Determining force applied to an ultrasonic sensor

ABSTRACT

In a method for determining force applied to an ultrasonic sensor, ultrasonic signals are emitted from an ultrasonic sensor. A plurality of reflected ultrasonic signals from a finger interacting with the ultrasonic sensor is captured. A first data based at least in part on a first reflected ultrasonic signal of the plurality of reflected ultrasonic signals is compared with a second data based at least in part on a second reflected ultrasonic signal of the plurality of reflected ultrasonic signals. A deformation of the finger during interaction with the ultrasonic sensor is determined based on differences between the first data based at least in part on the first reflected ultrasonic signal and the second data based at least in part on the second reflected ultrasonic signal. A force applied by the finger to the ultrasonic sensor is determined based at least in part on the deformation.

RELATED APPLICATIONS

This application is a continuation of and claims priority to and benefitof co-pending U.S. patent application Ser. No. 16/392,161 filed on Apr.23, 2019, entitled “DETERMINING FORCE APPLIED TO AN ULTRASONIC SENSOR”by Medina et al., having Attorney Docket No. IVS-671C, and assigned tothe assignee of the present application, the disclosure of which ishereby incorporated herein by reference in its entirety.

The patent application with application Ser. No. 16/392,161 is acontinuation of and claims priority to and benefit of then co-pendingU.S. patent application Ser. No. 15/449,770 filed on Mar. 3, 2017,entitled “DETERMINING FORCE APPLIED TO AN ULTRASONIC SENSOR” by Medinaet al., having Attorney Docket No. IVS-671, and assigned to the assigneeof the present application, the disclosure of which is herebyincorporated herein by reference in its entirety.

The patent application with application Ser. No. 15/449,770 claimspriority to and the benefit of then co-pending U.S. Patent ProvisionalPatent Application 62/302,886, filed on Mar. 3, 2016, entitled “FORCETOUCH SENSOR USING ULTRASONIC IMAGING,” by Medina et al., havingAttorney Docket No. IVS-671-PR/225.143, and assigned to the assignee ofthe present application, which is incorporated herein by reference inits entirety.

The patent application with application Ser. No. 15/449,770 also claimspriority to and the benefit of then co-pending U.S. Provisional PatentApplication 62/344,061, filed on Jun. 1, 2016, entitled “FORCE TOUCHSENSOR USING ULTRASONIC IMAGING: NAVIGATION MODES,” by Fayolle et al.,having Attorney Docket No. 225.185/IVS-671-PR2, and assigned to theassignee of the present application, which is incorporated herein byreference in its entirety.

BACKGROUND

Electronic devices having touch screens that can detect the force withwhich the user touches the screen exist today in smartphones. With thesescreens the system can detect where the users touches the screen andwith which force. This enables additional options for the implementationof applications and the user interface. Implementing force detectionfunctionality into a touch screen requires an additional sensing layeron the entire screen of the electronic device, which requires additionalcomponentry as well as manufacturing steps and costs.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthe Description of Embodiments, illustrate various embodiments of thesubject matter and, together with the Description of Embodiments, serveto explain principles of the subject matter discussed below. Unlessspecifically noted, the drawings referred to in this Brief Descriptionof Drawings should be understood as not being drawn to scale. Herein,like items are labeled with like item numbers.

FIG. 1 is a block diagram of an example mobile electronic device 100upon which embodiments described herein may be implemented.

FIG. 2 illustrates an example ultrasonic transducer system with phasedelayed transmission, according to some embodiments.

FIG. 3 illustrates another example ultrasonic transducer system withphase delayed transmission, according to some embodiments.

FIG. 4 illustrates an example phase delay pattern for a 9×]ultrasonictransducer array position, according to some embodiments.

FIG. 5 illustrates an example phase delay pattern for ultrasonic signaltransmission of a 9×]ultrasonic transducer block of a two-dimensionalarray of ultrasonic transducers, according to some embodiments.

FIG. 6 illustrates an example ultrasonic transducer system with phasedelayed transmission, according to some embodiments.

FIGS. 7A and 7B illustrate cross section views of an example ultrasonicsensor and a finger, according to some embodiments.

FIG. 7C illustrates examples of images of a finger taken at differentdepths, according to some embodiments.

FIG. 8 illustrates a flow diagram of an example method for capturingdata based on reflected ultrasonic signals, according to variousembodiments.

FIG. 9 illustrates a flow diagram of an example method for determiningforce applied to an ultrasonic sensor, according to various embodiments.

FIGS. 10A and 10B illustrate cross section views of an exampleridge/valley pattern of a finger at different forces, according to someembodiments.

FIG. 11 illustrates a flow diagram of an example method for combiningdata from an ultrasonic sensor with data from another sensor, accordingto various embodiments.

FIGS. 12A and 12B illustrate cross section views of translation of afinger relative an example ultrasonic sensor at different forces,according to some embodiments.

FIG. 13 illustrates a flow diagram of an example method for using theforce to provide navigation functionality on a display of an electronicdevice, according to various embodiments.

DESCRIPTION OF EMBODIMENTS

The following Description of Embodiments is merely provided by way ofexample and not of limitation. Furthermore, there is no intention to bebound by any expressed or implied theory presented in the precedingbackground or brief summary, or in the following detailed description.

Reference will now be made in detail to various embodiments of thesubject matter, examples of which are illustrated in the accompanyingdrawings. While various embodiments are discussed herein, it will beunderstood that they are not intended to limit to these embodiments. Onthe contrary, the presented embodiments are intended to coveralternatives, modifications and equivalents, which may be includedwithin the spirit and scope the various embodiments as defined by theappended claims. Furthermore, in this Description of Embodiments,numerous specific details are set forth in order to provide a thoroughunderstanding of embodiments of the present subject matter. However,embodiments may be practiced without these specific details. In otherinstances, well known methods, procedures, components, and circuits havenot been described in detail as not to unnecessarily obscure aspects ofthe described embodiments.

Notation and Nomenclature

Some portions of the detailed descriptions which follow are presented interms of procedures, logic blocks, processing and other symbolicrepresentations of operations on data within an electrical circuit.These descriptions and representations are the means used by thoseskilled in the data processing arts to most effectively convey thesubstance of their work to others skilled in the art. In the presentapplication, a procedure, logic block, process, or the like, isconceived to be one or more self-consistent procedures or instructionsleading to a desired result. The procedures are those requiring physicalmanipulations of physical quantities. Usually, although not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated in an electronic device.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the followingdiscussions, it is appreciated that throughout the description ofembodiments, discussions utilizing terms such as “emitting,”“capturing,” “comparing,” “determining,” “using,” “providing,” or thelike, refer to the actions and processes of an electronic device such asan electrical circuit.

Embodiments described herein may be discussed in the general context ofprocessor-executable instructions residing on some form ofnon-transitory processor-readable medium, such as program modules,executed by one or more computers or other devices. Generally, programmodules include routines, programs, objects, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. The functionality of the program modules may becombined or distributed as desired in various embodiments.

In the figures, a single block may be described as performing a functionor functions; however, in actual practice, the function or functionsperformed by that block may be performed in a single component or acrossmultiple components, and/or may be performed using hardware, usingsoftware, or using a combination of hardware and software. To clearlyillustrate this interchangeability of hardware and software, variousillustrative components, blocks, modules, logic, circuits, and stepshave been described generally in terms of their functionality. Whethersuch functionality is implemented as hardware or software depends uponthe particular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present disclosure. Also, the example fingerprint sensingsystem and/or mobile electronic device described herein may includecomponents other than those shown, including well-known components.

Various techniques described herein may be implemented in hardware,software, firmware, or any combination thereof, unless specificallydescribed as being implemented in a specific manner. Any featuresdescribed as modules or components may also be implemented together inan integrated logic device or separately as discrete but interoperablelogic devices. If implemented in software, the techniques may berealized at least in part by a non-transitory processor-readable storagemedium comprising instructions that, when executed, perform one or moreof the methods described herein. The non-transitory processor-readabledata storage medium may form part of a computer program product, whichmay include packaging materials.

The non-transitory processor-readable storage medium may comprise randomaccess memory (RAM) such as synchronous dynamic random access memory(SDRAM), read only memory (ROM), non-volatile random access memory(NVRAM), electrically erasable programmable read-only memory (EEPROM),FLASH memory, other known storage media, and the like. The techniquesadditionally, or alternatively, may be realized at least in part by aprocessor-readable communication medium that carries or communicatescode in the form of instructions or data structures and that can beaccessed, read, and/or executed by a computer or other processor.

Various embodiments described herein may be executed by one or moreprocessors, such as one or more motion processing units (MPUs), sensorprocessing units (SPUs), host processor(s) or core(s) thereof, digitalsignal processors (DSPs), general purpose microprocessors, applicationspecific integrated circuits (ASICs), application specific instructionset processors (ASIPs), field programmable gate arrays (FPGAs), aprogrammable logic controller (PLC), a complex programmable logic device(CPLD), a discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein, or other equivalent integrated or discrete logiccircuitry. The term “processor,” as used herein may refer to any of theforegoing structures or any other structure suitable for implementationof the techniques described herein. As it employed in the subjectspecification, the term “processor” can refer to substantially anycomputing processing unit or device comprising, but not limited tocomprising, single-core processors; single-processors with softwaremultithread execution capability; multi-core processors; multi-coreprocessors with software multithread execution capability; multi-coreprocessors with hardware multithread technology; parallel platforms; andparallel platforms with distributed shared memory. Moreover, processorscan exploit nano-scale architectures such as, but not limited to,molecular and quantum-dot based transistors, switches and gates, inorder to optimize space usage or enhance performance of user equipment.A processor may also be implemented as a combination of computingprocessing units.

In addition, in some aspects, the functionality described herein may beprovided within dedicated software modules or hardware modulesconfigured as described herein. Also, the techniques could be fullyimplemented in one or more circuits or logic elements. A general purposeprocessor may be a microprocessor, but in the alternative, the processormay be any conventional processor, controller, microcontroller, or statemachine. A processor may also be implemented as a combination ofcomputing devices, e.g., a combination of an SPU/MPU and amicroprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with an SPU core, MPU core, or any othersuch configuration.

Overview of Discussion

Discussion begins with a description of an example mobile electronicdevice with which or upon which various embodiments described herein maybe implemented. Example operations of a two-dimensional array ofultrasonic transducers are then described. Example force determinationand navigation using an ultrasonic sensor are then described.

Example Mobile Electronic Device

Turning now to the figures, FIG. 1 is a block diagram of an examplemobile electronic device 100. As will be appreciated, mobile electronicdevice 100 may be implemented as a device or apparatus, such as ahandheld mobile electronic device. For example, such a mobile electronicdevice may be, without limitation, a mobile telephone phone (e.g.,smartphone, cellular phone, a cordless phone running on a local network,or any other cordless telephone handset), a wired telephone (e.g., aphone attached by a wire), a personal digital assistant (PDA), a videogame player, video game controller, a navigation device, an activity orfitness tracker device (e.g., bracelet, clip, band, or pendant), a smartwatch or other wearable device, a mobile internet device (MID), apersonal navigation device (PND), a digital still camera, a digitalvideo camera, a portable music player, a portable video player, aportable multi-media player, a remote control, or a combination of oneor more of these devices.

As depicted in FIG. 1, mobile electronic device 100 may include a hostprocessor 110, a host bus 120, a host memory 130, a display device 140,and a sensor processing unit 170. Some embodiments of mobile electronicdevice 100 may further include one or more of an interface 150, atransceiver 160 (all depicted in dashed lines) and/or other components.In various embodiments, electrical power for mobile electronic device100 is provided by a mobile power source such as a battery (not shown),when not being actively charged.

Host processor 110 can be one or more microprocessors, centralprocessing units (CPUs), DSPs, general purpose microprocessors, ASICs,ASIPs, FPGAs or other processors which run software programs orapplications, which may be stored in host memory 130, associated withthe functions and capabilities of mobile electronic device 100.

Host bus 120 may be any suitable bus or interface to include, withoutlimitation, a peripheral component interconnect express (PCIe) bus, auniversal serial bus (USB), a universal asynchronousreceiver/transmitter (UART) serial bus, a suitable advancedmicrocontroller bus architecture (AMBA) interface, an Inter-IntegratedCircuit (I2C) bus, a serial digital input output (SDIO) bus, a serialperipheral interface (SPI) or other equivalent. In the embodiment shown,host processor 110, host memory 130, display 140, interface 150,transceiver 160, sensor processing unit (SPU) 170, and other componentsof mobile electronic device 100 may be coupled communicatively throughhost bus 120 in order to exchange commands and data. Depending on thearchitecture, different bus configurations may be employed as desired.For example, additional buses may be used to couple the variouscomponents of mobile electronic device 100, such as by using a dedicatedbus between host processor 110 and memory 130.

Host memory 130 can be any suitable type of memory, including but notlimited to electronic memory (e.g., read only memory (ROM), randomaccess memory, or other electronic memory), hard disk, optical disk, orsome combination thereof. Multiple layers of software can be stored inhost memory 130 for use with/operation upon host processor 110. Forexample, an operating system layer can be provided for mobile electronicdevice 100 to control and manage system resources in real time, enablefunctions of application software and other layers, and interfaceapplication programs with other software and functions of mobileelectronic device 100. Similarly, a user experience system layer mayoperate upon or be facilitated by the operating system. The userexperience system may comprise one or more software application programssuch as menu navigation software, games, device function control,gesture recognition, image processing or adjusting, voice recognition,navigation software, communications software (such as telephony orwireless local area network (WLAN) software), and/or any of a widevariety of other software and functional interfaces for interaction withthe user can be provided. In some embodiments, multiple differentapplications can be provided on a single mobile electronic device 100,and in some of those embodiments, multiple applications can runsimultaneously as part of the user experience system. In someembodiments, the user experience system, operating system, and/or thehost processor 110 may operate in a low-power mode (e.g., a sleep mode)where very few instructions are processed. Such a low-power mode mayutilize only a small fraction of the processing power of a full-powermode (e.g., an awake mode) of the host processor 110.

Display 140, may be a liquid crystal device, (organic) light emittingdiode device, or other display device suitable for creating and visiblydepicting graphic images and/or alphanumeric characters recognizable toa user. Display 140 may be configured to output images viewable by theuser and may additionally or alternatively function as a viewfinder forcamera.

Interface 150, when included, can be any of a variety of differentdevices providing input and/or output to a user, such as audio speakers,touch screen, real or virtual buttons, joystick, slider, knob, printer,scanner, computer network I/O device, other connected peripherals andthe like.

Transceiver 160, when included, may be one or more of a wired orwireless transceiver which facilitates receipt of data at mobileelectronic device 100 from an external transmission source andtransmission of data from mobile electronic device 100 to an externalrecipient. By way of example, and not of limitation, in variousembodiments, transceiver 160 comprises one or more of: a cellulartransceiver, a wireless local area network transceiver (e.g., atransceiver compliant with one or more Institute of Electrical andElectronics Engineers (IEEE) 802.11 specifications for wireless localarea network communication), a wireless personal area networktransceiver (e.g., a transceiver compliant with one or more IEEE 802.15specifications for wireless personal area network communication), and awired a serial transceiver (e.g., a universal serial bus for wiredcommunication).

Mobile electronic device 100 also includes a general purpose sensorassembly in the form of integrated SPU 170 which includes sensorprocessor 172, memory 176, an ultrasonic sensor 178, and a bus 174 forfacilitating communication between these and other components of SPU170. In some embodiments, SPU 170 may include at least one sensor 180(shown as sensor 180-1, 180-2, . . . 180-n) communicatively coupled tobus 174. In some embodiments, all of the components illustrated in SPU170 may be embodied on a single integrated circuit. It should beappreciated that SPU 170 may be manufactured as a stand-alone unit(e.g., an integrated circuit), that may exist separately from a largerelectronic device.

Sensor processor 172 can be one or more microprocessors, CPUs, DSPs,general purpose microprocessors, ASICs, ASIPs, FPGAs or other processorswhich run software programs, which may be stored in memory 176,associated with the functions of SPU 170.

Bus 174 may be any suitable bus or interface to include, withoutlimitation, a peripheral component interconnect express (PCIe) bus, auniversal serial bus (USB), a universal asynchronousreceiver/transmitter (UART) serial bus, a suitable advancedmicrocontroller bus architecture (AMBA) interface, an Inter-IntegratedCircuit (I2C) bus, a serial digital input output (SDIO) bus, a serialperipheral interface (SPI) or other equivalent. Depending on thearchitecture, different bus configurations may be employed as desired.In the embodiment shown, sensor processor 172, memory 176, sensor 178,and other components of SPU 170 may be communicatively coupled throughbus 174 in order to exchange data.

Memory 176 can be any suitable type of memory, including but not limitedto electronic memory (e.g., read only memory (ROM), random accessmemory, or other electronic memory). Memory 176 may store algorithms orroutines or other instructions for processing data received fromultrasonic sensor 178 and/or one or more sensor 180, as well as thereceived data either in its raw form or after some processing. Suchalgorithms and routines may be implemented by sensor processor 172and/or by logic or processing capabilities included in ultrasonic sensor178 and/or sensor 180.

A sensor 180 may comprise, without limitation: a temperature sensor, ahumidity sensor, an atmospheric pressure sensor, an infrared sensor, aradio frequency sensor, a navigation satellite system sensor (such as aglobal positioning system receiver), an acoustic sensor (e.g., amicrophone), an inertial or motion sensor (e.g., a gyroscope,accelerometer, or magnetometer) for measuring the orientation or motionof the sensor in space, or other type of sensor for measuring otherphysical or environmental quantities. In one example, sensor 180-1 maycomprise an acoustic sensor, sensor 180-2 may comprise a second acousticsensor, and sensor 180-n may comprise a motion sensor.

In some embodiments, ultrasonic sensor 178 and/or one or more sensors180 may be implemented using a microelectromechanical system (MEMS) thatis integrated with sensor processor 172 and one or more other componentsof SPU 170 in a single chip or package. Although depicted as beingincluded within SPU 170, one, some, or all of ultrasonic sensor 178and/or one or more sensors 180 may be disposed externally to SPU 170 invarious embodiments.

Example Operation of a Two-Dimensional Array of Ultrasonic Transducers

FIG. 2 illustrates an example ultrasonic transducer system 200 withphase delayed transmission, according to some embodiments. Asillustrated, FIG. 2 shows ultrasonic beam transmission and receptionusing a one-dimensional, five-element, ultrasonic transducer system 200having phase delayed inputs 210. In various embodiments, ultrasonictransducer system 200 is comprised of PMUT devices having a centerpinned membrane. In one embodiment, ultrasonic transducer system 200 isultrasonic sensor 178 of FIG. 1.

As illustrated, ultrasonic transducer system 200 includes fiveultrasonic transducers 202 including a piezoelectric material andactivating electrodes that are covered with a continuous stiffeninglayer 204 (e.g., a mechanical support layer). Stiffening layer 204contacts acoustic coupling layer 206, and in turn is covered by a platenlayer 208. In various embodiments, the stiffening layer 204 can besilicon, and the platen layer 208 formed from glass, sapphire, orpolycarbonate or similar durable plastic. It should be appreciated thatstiffening layer 204 and platen layer 208 are optional. For example,acoustic coupling layer 206 can operate as a single layer providing fortransmission of ultrasonic signals and serve as a contact surface forultrasonic transducer system 200. In another embodiment, acousticcoupling layer 206 can provide for transmission of ultrasonic signals inconjunction with one of stiffening layer 204 and platen layer 208.

The intermediately positioned acoustic coupling layer 206 can be formedfrom a plastic, epoxy, or gel such as polydimethylsiloxane (PDMS) orother material. In one embodiment, the material of acoustic couplinglayer 206 has an acoustic impedance selected to be between the acousticimpedance of layers 204 and 208. In one embodiment, the material ofacoustic coupling layer 206 has an acoustic impedance selected to beclose the acoustic impedance of platen layer 208, to reduce unwantedacoustic reflections and improve ultrasonic beam transmission andsensing. However, alternative material stacks to the one shown in FIG. 2may be used and certain layers may be omitted, provided the mediumthrough which transmission occurs passes signals in a predictable way.

In operation, and as illustrated in FIG. 2, the ultrasonic transducers202 labelled with an “x” are triggered to emit ultrasonic waves at aninitial time. At a second time, (e.g., 1-100 nanoseconds later), theultrasonic transducers 202 labelled with a “y” are triggered. At a thirdtime (e.g., 1-100 nanoseconds after the second time) the ultrasonictransducer 202 labelled with a “z” is triggered. The ultrasonic wavesinterfere transmitted at different times cause interference with eachother, effectively resulting in a single high intensity beam 220 thatexits the platen layer 208, contacts objects, such as a finger (notshown), that contact the platen layer 208, and is in part reflected backto the ultrasonic transducers. In one embodiment, the ultrasonictransducers 202 are switched from a transmission mode to a receptionmode, allowing the “z” ultrasonic transducer to detect any reflectedsignals 222. In other words, the phase delay pattern of the ultrasonictransducers 202 is symmetric about the focal point where high intensitybeam 220 exits platen layer 208.

It should be appreciated that in accordance with various embodiments,high intensity beam 220 can be formed to have a focal point at differentdistances above ultrasonic transducers 202. In one embodiment, highintensity beam 220 can be formed to have a focal point at the outersurface of platen layer 208 for capturing a pixel of an objectinteracting with ultrasonic transducer system 200. For example, highintensity beam 220 can be formed to capture pixels of an image of theouter surface of the epidermis of a finger (e.g., a fingerprint). Inother embodiments, high intensity beam 220 can be formed to have a focalpoint beyond the outer surface of platen layer 208 for capturing a pixelof an object interacting with ultrasonic transducer system 200. Forexample, high intensity beam 220 can be formed to capture pixels of animage of a predetermined depth within the dermis of a finger (e.g., adeeper layer within the finger). According the various embodiments, beamforming can be used to capture images of various depths into a fingerinteracting with ultrasonic transducer system 200. While beam forming isone method for capturing different imaging depths of an objectinteracting with ultrasonic transducer system 200, it should beappreciated that other methods for capturing images at differentinternal depths may also be used in accordance with the describedembodiments.

It should be appreciated that an ultrasonic transducer 202 of ultrasonictransducer system 200 may be used to transmit and/or receive anultrasonic signal, and that the illustrated embodiment is a non-limitingexample. The received signal (e.g., generated based on reflections,echoes, etc. of the acoustic signal from an object contacting or abovethe platen layer 208) can then be analyzed. As an example, an image ofthe object, a distance of the object from the sensing component,acoustic impedance of the object, a motion of the object, etc., can allbe determined based on comparing a frequency, amplitude, phase and/orarrival time of the received signal with a frequency, amplitude, phaseand/or transmission time of the transmitted acoustic signal. If theobject is, for example, a finger, various characteristics (e.g. densityand/or other acoustic properties) of the layers and features of thefinger may be determined. Moreover, results generated can be furtheranalyzed or presented to a user via a display device (not shown).

FIG. 3 illustrates another example ultrasonic transducer system 300 withphase delayed transmission, according to some embodiments. Asillustrated, FIG. 3 shows ultrasonic beam transmission and receptionusing a virtual block of two-dimensional, 24-element, ultrasonictransducers that form a subset of a 40-element ultrasonic transducersystem 300 having phase delayed inputs. In operation, an array position330 (represented by the dotted line), also referred to herein as avirtual block, includes columns 320, 322 and 324 of ultrasonictransducers 302. At an initial time, columns 320 and 324 of arrayposition 330 are triggered to emit ultrasonic waves at an initial time.At a second time (e.g., several nanoseconds later), column 322 of arrayposition 330 is triggered. The ultrasonic waves interfere with eachother, substantially resulting in emission of a high intensityultrasonic wave centered on column 322. In one embodiment, theultrasonic transducers 302 in columns 320 and 324 are switched off,while column 322 is switched from a transmission mode to a receptionmode, allowing detection of any reflected signals.

In one embodiment, after the activation of ultrasonic transducers 302 ofarray position 330, ultrasonic transducers 302 of another array position332, comprised of columns 324, 326, and 328 of ultrasonic transducers302 are triggered in a manner similar to that described in the foregoingdescription of array position 330. In one embodiment, ultrasonictransducers 302 of another array position 332 are activated after adetection of a reflected ultrasonic signal at column 322 of arrayposition 330. It should be appreciated that while movement of the arrayposition by two columns of ultrasonic transducers is illustrated,movement by one, three, or more columns rightward or leftward iscontemplated, as is movement by one or more rows, or by movement by bothsome determined number of rows and columns. In various embodiments,successive array positions can be either overlapping in part, or can bedistinct. In some embodiments the size of array positions can be varied.In various embodiments, the number of ultrasonic transducers 302 of anarray position for emitting ultrasonic waves can be larger than thenumber of ultrasonic transducers 302 of an array position for ultrasonicreception. In still other embodiments, array positions can be square,rectangular, ellipsoidal, circular, or more complex shapes such ascrosses.

Example ultrasonic transducer system 300 is operable to beamform a lineof a high intensity ultrasonic wave centered over column 322. It shouldbe appreciated that the principles illustrated in FIG. 3 for beamforminga line using columns of ultrasonic transducers is applicable toembodiments for beamforming a point using ultrasonic transducers, aswill be explained below. For instance, example ultrasonic transducersystem 300 includes columns of ultrasonic transducers in which theultrasonic transducers of each column are jointly operated to activateat the same time, operating to beamform along a line. It should beappreciated that the ultrasonic transducers of a two-dimensional arraymay be independently operable, and used for beamform points as well, aswill be described below. Moreover, it should be appreciated thatultrasonic transducer system 300 is able to beamform a line to aparticular distance over column 322. Similarly, an ultrasonic transducersystem for beamforming points may beamform a point to a particulardistance about the ultrasonic transducer system. In such a manner, aline or a point may be formed to have a focal point at different depthsinto an object interacting with an ultrasonic transducer system.

FIG. 4 illustrates example two-dimensional array 400 of ultrasonictransducers according to an embodiment. FIG. 4 illustrates phase delaypattern 410, indicating ultrasonic transducers that are activated forforming a beam to a point at the center of phase delay pattern 410. Asillustrated, phase delay pattern 410 is a three phase (indicated usingdifferent hatch patterns) activated phase delay pattern of ultrasonictransducers in a 9×9 array position that is used to generate anultrasonic beam with a focus point at the center of phase delay pattern410 and having a particular depth above two-dimensional array 400.

FIG. 5 illustrates an example phase delay pattern 500 for ultrasonicsignal transmission of a 9×]ultrasonic transducer block of atwo-dimensional array of ultrasonic transducers, according to someembodiments. For example, phased delay pattern 500 may be used as phasedelay pattern 410 of FIG. 4. As illustrated in FIG. 5, each number inthe ultrasonic transducer array is equivalent to the nanosecond delayused during operation, and an empty element (e.g., no number) in phasedelay pattern 500 means that an ultrasonic transducer is not activatedfor signal transmission during operation. In various embodiments,ultrasonic wave amplitude can be the same or similar for each activatedultrasonic transducer, or can be selectively increased or decreasedrelative to other ultrasonic transducers. In the illustrated pattern,initial ultrasonic transducer activation is limited to ultrasonictransducers in the corners of phase delay pattern 500, followed 10nanoseconds later by a rough ring of ultrasonic transducers around theedges of phase delay pattern 500. After 23 nanoseconds, an interior ringof ultrasonic transducers is activated of phase delay pattern 500.Together, the twenty-four activated ultrasonic transducers generate anultrasonic beam centered on phase delay pattern 500 and focused to aparticular depth above the 9×]ultrasonic transducer block. In otherwords, phase delay pattern 500 is symmetric about the focal point wherea high intensity beam contacts or penetrates an object.

It should be appreciated that different ultrasonic transducers of phasedelay pattern 500 may be activated for receipt of reflected ultrasonicsignals. For example, the center 3x3 ultrasonic transducers of phasedelay pattern 500 may be activated to receive the reflected ultrasonicsignals. In another example, the ultrasonic transducers used to transmitthe ultrasonic signal are also used to receive the reflected ultrasonicsignal. In another example, the ultrasonic transducers used to receivethe reflected ultrasonic signals include at least one of the ultrasonictransducers also used to transmit the ultrasonic signals.

FIG. 6 illustrates an example ultrasonic transducer system 600 withphase delayed transmission, according to some embodiments. FIG. 6 showsfive different modes of ultrasonic beam transmission using an exampleone-dimensional, fifteen-element, ultrasonic transducer system 600having phase delayed inputs. As illustrated, ultrasonic transducers 602can be operated in various modes to provide ultrasonic beam spotsfocused along line 650 (e.g., a top of a platen layer or a depthpenetrating into an object). In a first mode, a single ultrasonictransducer 652 is operated to provide a single broad ultrasonic beamhaving a peak amplitude centered on arrow 653. In a second mode,multiple ultrasonic transducers in a symmetrical pattern 654 about thecenter ultrasonic transducer are sequentially triggered to emitultrasonic waves at differing initial times. As illustrated, a centerlocated transducer is triggered at a delayed time with respect tosurrounding transducers (which are triggered simultaneously). Theultrasonic waves interfere with each other, resulting in a single highintensity beam 655. In a third mode, for ultrasonic transducers 656located adjacent to or near an edge of the ultrasonic transducer system600, an asymmetrical triggering pattern can be used to produce beam 657.In a fourth mode, asymmetrical triggering patterns for transducers 658can be used to steer an ultrasound beam to an off-center location 659. Ashown, the focused beam 659 can be directed to a point above and outsideboundaries of the ultrasonic transducer system 600. In a fifth mode, thebeam can be steered to focus at a series of discrete positions, with thebeam spacing having a pitch less than, equal to, or greater than a pitchof the ultrasonic transducers. In FIG. 6, transducers 660 are triggeredat separate times to produce beam spots separated by a pitch less thanthat of the ultrasonic transducers (indicated respectively by solidlines directed to form beam spot 661 and dotted lines to form beam spot663). It should be appreciated that in accordance with variousembodiments ultrasonic transducers of ultrasonic transducer system 600can be arranged into blocks of ultrasonic transducers, wherein eachblock of ultrasonic transducers is collectively controllable.

Example Force Determination and Navigation Using an Ultrasonic Sensor

Embodiments described herein provide for the integration of anultrasound sensor (also referred to herein as an “ultrasonic sensor” oran “ultrasonic imaging sensor”) in a mobile device. In variousembodiments, the ultrasonic sensor may be capable of capturing thefingerprint of a user of the device.

FIGS. 7A and 7B illustrate cross section views of an example ultrasonicsensor 730 and a finger 710, according to some embodiments. Withreference to FIG. 7A, finger 710 is shown interacting with ultrasonicsensor 730. It should be appreciated that the dimensions of ultrasonicsensor 730 may be chosen to capture only a small section of thefingerprint of finger 710, or the dimensions of ultrasonic sensor 730may be chosen larger to capture substantially the complete fingerprint.In one embodiment, a cover 720 overlies ultrasonic sensor 730. Invarious embodiments, cover 720 may be made of transparent material,e.g., a thin sheet of glass, or other thin opaque materials, such as,but not limited to plastic, resin, rubber, Teflon, epoxy, glass,aluminum-based alloys, sapphire, titanium nitride (TiN), Silicon carbide(SiC), diamond, etc. For example, cover 720 may provide protection forultrasonic sensor 730 by preventing a user from coming into contact withultrasonic sensor 730. It should be appreciated that ultrasonic sensor730 may be in direct contact with cover 720, or there may be a gapseparating ultrasonic sensor 730 and cover 720. In various embodiments,the gap may be filled with an acoustic coupling material including air,solid liquid, gel-like materials, or other materials for supportingtransmission of acoustic signals.

Ultrasonic sensor 730 may be incorporated on the different exteriorfaces of an electronic device (e.g., mobile electronic device 100 ofFIG. 1), depending on the ergonomics and easy for the user to interactwith ultrasonic sensor 730 using a finger 710. For example, if theelectronic device includes a display, ultrasonic sensor 730 may beincluded in the same side as the display, behind the display, on an edgeof the electronic device, or on the back of the electronic device. Inaccordance with some embodiments, ultrasonic sensor 730 may beincorporated in a button of the electronic device. In some embodiments,visual or textural markers may be present on cover 720 to indicate tothe user where ultrasonic sensor 730 is positioned and where to putfinger 710.

Ultrasonic sensor 730 may provide multiple functionalities. Forinstance, in addition to being operable capture the fingerprint of theuser, ultrasonic sensor 730 may also be used to determine the forceapplied by the user (e.g., the force of finger 710 applied to ultrasonicsensor 730), and may further be used to provide navigationalfunctionality. The different functionalities or modes may be selectedand/or activated automatically, for example, depending on the context orapplication of the device, and the different functionalities or modesmay be adaptive to the user and the user's habits or preferences. In theforce detection mode, the sensor may require more power and processingresources, and therefore the force detection mode may only be activatedwhen useful. In some embodiments, the parameters of the force detectionprocess may be adapted to use less power or computing resources, whichmay come at the costs of quality or confidence in the determined force.In some embodiments, the force detection process may be disabled basedon the available power or computing resources. Embodiments describedherein pertain to methods to derive the applied force and methods fornavigation.

Ultrasonic sensor 730 is operable to emit and detect ultrasonic waves(also referred to as ultrasonic signals or ultrasound signals). Theemitted ultrasonic waves are reflected from any objects in front ofultrasonic sensor 730, and these reflected ultrasonic waves, or echoes,are then detected. Where the object is a finger (e.g., finger 710), thewaves are reflected from different features of the finger, such as thesurface features (e.g., surface features 712 of FIG. 7A and surfacefeatures 722 of FIG. 7B) on the skin (e.g., the epidermis), or features(e.g., features 716 of FIG. 7A and surface features 726 of FIG. 7B)present in deeper layers of the finger (e.g., the dermis). Examples ofsurface features of a finger are ridges and valleys of a fingerprint.For example, the reflection of the sound waves from the ridge/valleypattern enables ultrasonic sensor 730 to produce a fingerprint imagethat may be used for identification of the user. In optical fingerprintsensors, the same principle of emission and reflection are used todetect the fingerprint. However, in contrast to the optical waves (ofvisible wavelengths) that reflect from the outside of the skin (surfacefeatures), the ultrasound waves may penetrate further into the skin ofthe finger and enable the capture of deeper layers and features.Therefore, ultrasonic sensor 730 is able to provide depth information,from which a multi-dimensional fingerprint may be determined, such ase.g. a 3D fingerprint.

It should be appreciated that the features that can reflect ultrasonicwaves, and used to determine deformation, may be any anatomical featurefrom the different layers of the finger, e.g., the epidermis layer, thedermis layer, or subcutaneous tissue. The features may be the layersitself, transitions between different layers, features within the layers(e.g., pores), or features traversing the layers (e.g., capillary bloodvessels). Which features may be used depends on the penetration depth ofthe ultrasound waves and the imaging resolution. The features need notdirectly be the anatomical features, but may be features of ultrasonicsignals caused by the anatomical features, such as specific reflectionsor absorptions of the signal.

In order to obtain the 3D fingerprint, the depth information is detectedusing ultrasonic sensor 730. The depth information can be obtained dueto the fact that the ultrasonic waves reflect from features at differentdepths in the skin. The reflection time, which is defined as the timebetween the emission of the ultrasonic waves and the detection of thereflected ultrasonic waves, increases as a function of the depth of thefeatures. Therefore, by analyzing the reflected waves as a function oftime, the features can be determined as a function of depth. Any one orcombination of the beam forming techniques discussed in relation to FIG.2 through FIG. 6 may be used for optimizing signals from a certain depthor from a certain layer. Images can be created that correspond to acertain depth within a finger. An array of images of different depthsmay be defined as the 3D fingerprint. Images may also be created tovisualize other cross sections of the finger, for example perpendicularto the cover surface or sensor surface. Fingerprints or 3D fingerprintmay not just be defined as images, but also as multi-dimensional datacorresponding to various (acoustic) properties of the finger (e.g.density, acoustic absorption, acoustic reflection).

FIG. 7C illustrates examples of images of a finger taken at differentdepths, according to some embodiments. Image 750 is an image of theepidermis of a finger and image 760 is an image of the dermis of thesame finger. Ridges 762 and valleys 764 illustrate the features ofimages 750 and 760, and how the features correspond to each other. Itshould be appreciated that in image 750 the valleys appear dark due toreflection at air interference and the ridges appear light due to theultrasonic wave passing into the epidermis, while in image 760 thevalleys appear light as they are shadowed by the epidermis valleys andthe ridges appear dark due to the reflection at the dermis.

As illustrated in FIGS. 7A and 7B, a finger 710 interacting withultrasonic sensor 730 and in contact with cover 720. FIGS. 7A and 7Bshow finger 710 contacting cover 720 with a different force, asillustrated by the different compression of dermal layers and featuresand a different size of a contact region. With reference to FIG. 7A,finger 710 is in contact with cover 720 at contact region 718, wherecontact region 718 defines the portion of the surface of finger 710 thatis in contact with cover 720. Similarly, with reference to FIG. 7B,finger 710 is in contact with cover 720 at contact region 728, wherecontact region 728 is larger than contact region 718. Moreover, dermallayers and features 716 of FIG. 7A are spaced farther apart than dermallayers and features 726 of FIG. 7B. Thus, finger 710 is contacting cover720 with a larger force in FIG. 7B than in FIG. 7A.

It should be appreciated that the force that is applied to cover 720 isnot instantaneous, but rather increases from when the user startstouching cover 720 until reaching a maximum, after which the force maydecrease if the user removes finger 710. Therefore, by obtaining thereflected ultrasonic signals as a function of time the change of depthof the features can be determined, and the force may be determined (as afunction of time). In one embodiment, a correction may be made for thedistance from ultrasonic sensor 730 to cover 720 that finger 710contacts. For instance, FIG. 7A may illustrate finger 710 just aftercontacting cover 720 and FIG. 7B may illustrate finger 710 contactingcover 720 at a maximum force, as illustrated by the increasedcompression of dermal layers and features 726 as compared to dermallayers and features 716 and the increased size of contact region 728 ascompared to contact region 718.

FIG. 8 illustrates a flow diagram 800 of an example method for capturingdata based on reflected ultrasonic signals, according to variousembodiments. Procedures of this method will be described with referenceto elements and/or components of various figures described herein. It isappreciated that in some embodiments, the procedures may be performed ina different order than described, that some of the described proceduresmay not be performed, and/or that one or more additional procedures tothose described may be performed. Flow diagram 800 includes someprocedures that, in various embodiments, are carried out by one or moreprocessors (e.g., host processor 110 or sensor processor 172 of FIG. 1)under the control of computer-readable and computer-executableinstructions that are stored on non-transitory computer-readable storagemedia. It is further appreciated that one or more procedures describedin flow diagram 800 may be implemented in hardware, or a combination ofhardware with firmware and/or software.

In one embodiment, at procedure 810 of flow diagram 800, an ultrasonicsensor (e.g., ultrasonic sensor 730 of FIGS. 7A and 7B) emits ultrasonicwaves that are reflected from features of the finger (e.g., finger 710of FIGS. 7A and 7B). It should be appreciated that the ultrasonic wavescan be reflected from surface features of the finger as well as deeperfeatures within the finger. In one embodiment, the ultrasonic waves areemitted along a predefined axis of the finger. In another embodiment,the ultrasonic waves are emitted across an area of the ultrasonicsensor, and the area may be of any shape or form, such as e.g. a square,a rectangle, a line, etc. In one embodiment, flow diagram 800 commenceswhen a finger is detected. The detection of a finger may be done byactivating only a small section of the ultrasonic sensor in order toreduce power demands. When the presence of a finger is detected, forexample by the detection of a ridge\valley pattern, the entireultrasonic sensor may be activated.

In one embodiment, as shown at procedure 812, the ultrasonic waves areemitted along a predefined axis. For example, the predefined axis isperpendicular to the surface of the ultrasonic sensor. In oneembodiment, as shown at procedure 814, the ultrasonic waves are emittedover at least a one-dimensional or two-dimensional block of ultrasonictransducers of the ultrasonic sensor. For example, ultrasonic signalsare emitted over an area of the ultrasonic sensor, such as a 2×2 orlarger grouping of reflected ultrasonic waves.

At procedure 820, the reflected ultrasonic waves are detected/capturedat the ultrasonic sensor. In one embodiment, as shown at procedure 822,the reflected ultrasonic waves are captured along a predefined axis. Forexample, the predefined axis is perpendicular to the surface of theultrasonic sensor. It should be appreciated that the predefined axis maybe defined relative to the ultrasonic sensor or may be defined relativeto the finger interacting with the ultrasonic sensor. For example, thepredefined axis may be a static set of ultrasonic transducers forcapturing ultrasonic signals (e.g., the ultrasonic transducers of theultrasonic sensor do not change). In another example, the predefinedaxis is static relative to the finger interacting with the ultrasonicsensor and dynamic relative to the ultrasonic sensor (e.g., as thefinger moves/rotates relative to the ultrasonic sensor, the ultrasonictransducers of the ultrasonic sensor that emit and capture the reflectedultrasonic signals move/rotate with the finger). A surface scan may beperformed, requiring less time and resources, to verify the position ofthe finger, and adjust, if needed the position of the axis in order tomaintain the position with respect to the finger. In one embodiment,where the predefined axis is static relative to the finger and thefinger moves/rotates relative to the ultrasonic sensor, procedure 810 isupdated to emit ultrasonic signals along the appropriate predefinedaxis.

In one embodiment, as shown at procedure 824, the reflected ultrasonicwaves are captured over at least a one-dimensional or two-dimensionalblock of ultrasonic transducers of the ultrasonic sensor. For example,ultrasonic signals are captured over an area of the ultrasonic sensor,such as a 2×2 or larger grouping of reflected ultrasonic waves.

At procedure 830, data is generated based at least in part on thereflected ultrasonic signals. In one embodiment, as shown at procedure832, a profile of at least one characteristic of the reflectedultrasonic signals is generated. In one embodiment, the at least onecharacteristic represents a characteristic of the reflected ultrasonicsignals (e.g., reflected signal strength). In one embodiment, the atleast one characteristic represents a characteristic of the fingertissue derived from the reflected ultrasonic signals, such as e.g.tissue density or signal absorption. In one embodiment, the at least onecharacteristic represents a feature of the finger derived from thereflected ultrasonic signals. In one embodiment, where the reflectedultrasonic signals are captured along a predefined axis, the datarepresents the at least one characteristic as a function of depth intothe finger. In one embodiment, the data represents the at least onecharacteristic as a function of depth for at least one position of thefinger. For example, the profile includes the depth for the center ofthe fingerprint and for ridges of the fingerprint.

In one embodiment, the depth information is extracted from the reflectedultrasonic waves. The different imaging depths of the individual layersthat are analyzed depend on the settings, such as the timing and theemitted power (e.g., 10 um, 20 um, 30 um, 40 um etc.) For example, forvery low depths, which substantially limit the info to the ridge/valleypattern, the timing intervals of the different layers are small andrelatively low power is required. For capturing the deeper layers, thetiming intervals may be larger, and also more power may be required.

In one embodiment, as shown at procedure 834, an image based on thereflected ultrasonic signals is generated. In one embodiment, the imageis generated for a particular depth, wherein the image illustratesfeatures of the finger at the particular depth. In one embodiment, asshown at procedure 836, a 3D fingerprint is generated based on theultrasonic signals. In one embodiment, the 3D fingerprint is generatedby generating images of the finger at different depths, and arrangingthem into an array. In one embodiment, the 3D fingerprint is determinedbased on the extracted layers and depth information.

In accordance with one embodiment, as shown at procedure 840, a user isidentified and/or authenticated based on the generated data. Forexample, where an image of a fingerprint is generated, the image of thefingerprint can be compared to a library of fingerprint images. If thegenerated image matches (e.g., within a confidence level) an image ofthe library, the user can be identified and/or authenticated.

In addition to providing more detailed fingerprint information, thedepth features may also be used to obtain the force or pressure that theuser applies with the finger. When the user applies more pressure orforce, the features and layers of the skin may be compressed ordeformed. For example, FIG. 7A shows an example of the user applying lowforce, while FIG. 7B shows an example of the user applying a higherforce. FIG. 7B illustrates that when a higher force is applied, acompression and deformation of the different layers occurs. This meansthat the features that can be detected by the sensor may become deformedand their depth position may change. The features can also be displaced.The change in depth position means that a particular feature may changefrom a first imaging depth to a second imaging depth, where the imagingdepths represent the imaging layers at a certain depth which arecontrolled through the timing of the reflected waves. When determiningthe depth, a correction may be made for the distance from the sensor tothe surface of the cover that the user contacts.

FIG. 9 illustrates a flow diagram of an example method for determiningforce applied to an ultrasonic sensor, according to various embodiments.Procedures of this method will be described with reference to elementsand/or components of various figures described herein. It is appreciatedthat in some embodiments, the procedures may be performed in a differentorder than described, that some of the described procedures may not beperformed, and/or that one or more additional procedures to thosedescribed may be performed. Flow diagram 900 includes some proceduresthat, in various embodiments, are carried out by one or more processors(e.g., host processor 110 or sensor processor 172 of FIG. 1) under thecontrol of computer-readable and computer-executable instructions thatare stored on non-transitory computer-readable storage media. It isfurther appreciated that one or more procedures described in flowdiagram 900 may be implemented in hardware, or a combination of hardwarewith firmware and/or software.

In one embodiment, at procedure 910 of flow diagram 900, an ultrasonicsensor (e.g., ultrasonic sensor 730 of FIGS. 7A and 7B) emits ultrasonicwaves that are reflected from features of the finger (e.g., finger 710of FIGS. 7A and 7B). Procedure 910 is performed in the same manner asprocedure 810 of FIG. 8.

At procedure 920, the reflected ultrasonic waves are detected/capturedat the ultrasonic sensor. Procedure 920 is performed in the same manneras procedure 820 of FIG. 8.

At procedure 930, a first data based at least in part on a firstreflected ultrasonic signal of the plurality of reflected ultrasonicsignals is compared with a second data based at least in part on asecond reflected ultrasonic signal of the plurality of reflectedultrasonic signals. In one embodiment, the first data and second datarepresent at least one characteristic of the reflected ultrasonicsignals. In one embodiment, the first data and second data are profilesof at least one characteristic of the reflected ultrasonic signals. Inone embodiment, the at least one characteristic represents a feature ofthe finger derived from the reflected ultrasonic signals. In oneembodiment, the at least one characteristic represents a characteristicof the tissue of the finger derived from the reflected ultrasonicsignals. In one embodiment, where the reflected ultrasonic signals arecaptured along a predefined axis, the data represents the at least onecharacteristic as a function of depth into the finger. In anotherembodiment, the first data and second data are images. In anotherembodiment, the first data and the second data are 3D fingerprints. Thefirst data and the second data may be derived from a single reflectedultrasonic signal. For example, the first data may be derived from afirst time interval in the reflected ultrasonic signal, and the seconddata may be derived from a second time interval in the reflectedultrasonic signal. The first data and the second data may also bederived from different reflected ultrasonic signal due to differentultrasonic signals emitted at different times. For example, the firstdata may be derived from a first reflected ultrasonic signal due to afirst emitted ultrasonic signal, and the second data may be derived froma second reflected ultrasonic signal due to a second emitted ultrasonicsignal.

At procedure 940, a deformation of the finger during interaction withthe ultrasonic sensor is determined based on differences between thefirst data based at least in part on the first reflected ultrasonicsignal and the second data based at least in part on the secondreflected ultrasonic signal. It should be appreciated that deformationof the finger may include compression of layers of the finger,compression of features of the finger, a change in curvature of at leastone layer of the finger, and/or displacement of features of the finger,individually or in any combination.

For instance, in one embodiment, as shown at procedure 942, adeformation of at least one dermal layer of the finger is determined.FIGS. 10A and 10B illustrate cross section views of an exampledeformation of the epidermis and dermal layers (as shown as features1010 of FIG. 10A and features 1020 of FIG. 10B) at different forces ofcontact with a cover of an ultrasonic sensor, according to someembodiments. As shown, as the contact pressure increases from FIG. 10Ato FIG. 10B, the features of the epidermis and dermal layers arecompressed and deformed as a result of the increase in applied force.Accordingly, the deformation of layers of the finger at various depthscan be determined based on the differences in the appearance of featuresof the layers over time due to a change of force over time.

In one embodiment, as shown at procedure 944, a change in the curvatureof at least one dermal layer is determined. With reference again toFIGS. 7A and 7B, the curvature of deeper layers 716 in finger 710differs from the curvature of the deeper layers 726 of finger 710. Dueto the higher force applied to finger 710 in FIG. 7B compared to FIG.7A, the curvature changes. As shown, the layers flatten, and thecurvature radius increases. Accordingly, the deformation of layers ofthe finger at various depths can be determined based on the differencesin the curvature of the layers over time.

In one embodiment, based on the extracted depth information, thepositions of the layers and/or features are determined as a function oftime. By comparing the currently derived position with one or moreprevious positions, the position change of the layer and/or feature canbe determined as a function of time. In addition, any detecteddeformation of the layers and/or features may be used. Based on thederived compression and/or deformation information, the force applied bythe user may be determined. Depending on the settings, the completesensor surface may be activated, or a smaller section of the sensor (aselection of ultrasonic transducers) may be activated. For example, ifidentifiable features are only present in one section of the finger,only the corresponding section of the sensor maybe activated, with anapplied margin to be sure to cover the features of interest.

At procedure 950, a force applied by the finger to the ultrasonic sensoris determined based at least in part on the deformation. In oneembodiment, as shown at procedure 952, at least one of a normalcomponent of the force and lateral component of the force is determined.A normal component of force represents the force applied directly intoultrasonic sensor (e.g., perpendicular to the ultrasonic sensor). Thenormal component of force can be determined based on the deformation ofthe finger (e.g., deformation of features of the epidermis and/or dermallayers of the finger). A lateral component of force represents the forceexerted by the finger parallel to the surface of the ultrasonic sensor.The lateral component of force can be determined based on the movementof features of the finger (e.g., features of the epidermis and/or dermallayers) relative to the surface of the ultrasonic sensor. In otherembodiments, the force may be determined along a predefined axis ordecomposed with respect to another reference frame, which may be tied tothe sensor or to the finger.

In one embodiment, as shown at procedure 954, a maximum force during atime window is determined. It should be appreciated that any interactionbetween a finger and an ultrasonic sensor can generate a force. Themaximum force within a time period (e.g., between a finger initiatedinteraction with the ultrasonic sensor and the finger completinginteraction with the ultrasonic sensor) is determined for providinginteractive application inputs, etc. The maximum force is the greatestforce applied, where the maximum force is determined as the greatestdeformation of features of the finger. In some embodiment, the maximumforce is determined by detecting a (local) maximum in the deformation.In order words, as soon as the force/deformation starts to decreases,the maximum force can be determined. In some embodiments, a force abovea certain threshold may be used to trigger and action, without waitingto achieve a maximum. This embodiment would have a decreased latency.

In one embodiment, as shown at procedure 956, at least one of amagnitude of the force and a direction of the force during a time windowis determined. The magnitude of the force and/or the direction of theforce can be determined based on the deformation of the finger (e.g.,deformation of features of the epidermis and/or dermal layers of thefinger) and/or the movement of features of the finger (e.g., features ofthe epidermis and/or dermal layers) relative to the surface of theultrasonic sensor.

In one embodiment, to determine the force, an absolute curvature of oneor more layers may be determined, and monitored over time.Alternatively, to determine the force, a relative change of curvature ofa first layer compared to a second layer may be determined, andmonitored over time. A calibration process may be used to determine therelation between the curvature and the force, and this calibrationprocess may also include a step to determine the best layer to use forthe force calculation.

In one embodiment, to determine the force, an absolute position of oneor more layers or features may be determined, and monitored over time.Alternatively, to determine the force, a relative change of a positionof a first layer/feature compared to a second layer/feature may bedetermined, and monitored over time. A calibration process may be usedto determine the relation between the position and the force, and thiscalibration process may also include a step to determine the best layeror features to use for the force calculation.

In some embodiments, a calibration process may be performed where theuser is asked to perform different interactions with the sensor,covering the range of force relevant for the user. The calibration maybe applied to one or more fingers. The calibration process may be usedto perform extensive scans in order to locate layers and features ofinterest to determine the force. The information may be used to optimizethe force detection process by knowing which features and layers workbest and give most reproducible results. This calibration process mayrequire a large amount of processing and power resources, but has thepurpose of optimizing the force detection process to use as little aspossible resources, while still having the required results. Inaccordance with various embodiments, the calibration process includes atleast one of: determining a maximum of the force applied by the finger,identifying a suitable feature of the finger to determine the force,identifying a suitable dermal layer to determine the force, anddetermining a relationship between the deformation and the force.

Imaging algorithms may be used to detect certain features and layers ingenerated data (e.g., depth profiles, images or 3D fingerprints) anddetermine how these features change in depth during the application ofthe force. In one embodiment, images of different depths may be producedby analyzing the reflected waves in certain time intervals, wherein eachtime interval corresponds to an image at a certain imaging depth. Whenthe force is applied, features may change from one image (depth) toanother image (depth). By detecting in which image a feature is detectedas a function of time, the force may be determined as a function oftime. In one example, the images may be numbered, and each numbercorresponds to a certain imaging depth, and the result of the imagealgorithm is an image number as a function of time, which is thenconverted to a depth as a function of time. Some features may bethree-dimensional and cover more than a single image. The change indepth may also be interpolated between different images, for example,resulting in non-integer results of the image numbers. Image analysis offeature of the finger may be used to determine deformation of thefeatures. For example, polynomial or other shape fitting algorithms maybe used to determine the shape of features, and then the change of theshape, or the parameters describing the shape, may be used to determinethe deformation.

In another embodiment, features may be determined directly in thereflected ultrasonic signals, without first converting the reflecteddata into images at different depth. In this algorithm, the reflectiontime of the detected feature is derived, and when the force changes,this reflection time changes. For example, when more force is applied,the reflection time decreases. In one example, the different pixelscaptured by the ultrasonic sensor may be grouped in order to facilitatethe processing of the data and determining the change, e.g., intensitypatterns of the grouped pixels. Patterns may be detected in thereflected ultrasonic signals, and the timing characteristics, or changein timing characteristics, of these patterns may be used to derive theforce. For example, the patterns may be compressed or the time delay ofthe patterns may change. This analysis may be done per pixel or pergroup of pixels. In one embodiment, the change or shift of the patternof the characteristics of the signal along a predefined axis may be usedto determine the force.

In some embodiments, features may be recognized for determining thedepth position of these features over time. In other embodiments, thealgorithms may be designed to determine a plurality of features, anddetermined the relative depth of these features with respect to eachother. For example, features of different depth would be pressed closertogether when more force is applied. In other embodiments, thealgorithms may be designed to analyze the deformation of layers and/orfeatures, and determine the force by the amount of deformation. In someembodiments, a first force may be determined based on the deformation offeatures, and a second force may be determined based on a change ofposition of feature or compression of layers. A weighted average of thefirst force and the second force may then be determined, where theweight may depend on, e.g., the confidence or quality of thecalculation. The weights may also depend on the user, or maybe whichfinger the user is using. Different combinations and weights may bebetter for some users of fingers of the users. These weights may bedetermined during calibration.

In some embodiments, the relation between the compression/deformationand the force may be determined as an average of a group of users andpredefined in the system. The average group of users may be adjusteddepending on demographics or user profiles. Alternatively, the relationmay be adaptive as the system will learn the range ofcompression/deformation for each user, and adapt the relationshipaccordingly. In other words, the maximal measured force/deformation maybe stored and used for future reference. The force may be given as asimple binary result, e.g., a low force or high force, or the force maybe given with a (predefined) number of levels. The number of levels maybe adaptively distributed evenly over the range of force of the user.

In some embodiments, the thermal coupling between the finger and theultrasonic sensor, and the change of the thermal coupling as a functionof the force or pressure may be used in addition to the force determinedfrom the deformation. For example, a finger and a surface of anultrasonic sensor can have a significantly different temperature.Physics of sound waves propagation in solid materials illustrate arelationship between temperature of traveled layers and the ultimatelycollected ultrasound signal. Young's modulus (thus sound velocity),scattering coefficients, interface impedances are examples of materialcharacteristics affecting the signal and that can significantly changewith temperature. This effect can be used to determine the “efficiency”of the thermal coupling, and thus finger pressure because the couplingincreases with increased pressure. Therefore, by deducing the thermalcoupling based on the ultra-sound signals, the applied pressure can bededuced. This information can be combined with information about thepressure obtained from other techniques describe here. The piezoelectricmaterial itself can have behavior changes with temperature. Temperaturesensors in the sensor itself or in the host device (e.g., a smartphone)may provide temperature information needed for the calculation of thepressure. Based on the temperature measurement and the thermal coupling,the expected behavior of the ultrasonic waves as a function of thetemperature and force can be predicted and/or modeled to improve theaccuracy of the measurements (or images), e.g., bias and/or driftcorrection as a function of temperature.

FIGS. 10A and 10B illustrate cross section views of an exampleridge/valley pattern of a finger at different forces, according to someembodiments. The detection of the features discussed above may belimited to the surface features of the finger, such as the actual ridgesand valleys of the surface of the finger. A depth analysis may belimited to approximately the height of the fingerprint structures. Bypressing the finger against the surface (e.g., cover) of the ultrasonicsensor, the ridge/valley pattern may be modified or compressed, whichcan then be detected and used to determine the applied force. Forexample, the air cavity due to the valleys may be decreased due to theapplied force, and the surface ratio of the ridges and valleys may bechanged. The shape of the ridges may also change due to the appliedforce, and this change may be determined through the depth analysisusing the sensor. Thus, the change of shape of the ridge and valleys dueto compression and/or deformation can be used to derive the appliedforce. In one example, the determined contact surface may be used toderive the applied force. FIG. 10A shows an example of the ridge/alleypattern at low-force, shown as features 1010, and FIG. 10B shows thesame pattern, shown as features 1020, at a higher force where thepattern is compressed leading to a greater contact surface and smallervalleys.

In some embodiments, the analysis of the ridge/valley pattern and theanalysis of deeper layers and/or features may be combined. In someembodiments, each analysis may be performed separately, and then theresults may be combined, for example, by averaging or weightedaveraging. The applied weight may depend on the obtained results and aconfidence factor. The different algorithms may produce a confidencefactor of the determined force, and the higher the confidence factor thehigher the weight in the averaging. In other embodiments, the differentalgorithms may also be performed sequentially. For example, a firstalgorithm may determine the force, and a certain confidence factor. Thesecond algorithm may only be used in case the confidence factor is belowa preset threshold. For each analysis the active section of the sensormay be adapted, for example, only a central section of the sensor may beused. In one embodiment, results from the surface analysis to determinethe force may help determine the best location to perform an in depthanalysis.

The data from the ultrasonic sensor may also be combined with data fromother sensors to deduce the force of the pressing action of the user.FIG. 11 illustrates a flow diagram 1100 of an example method forcombining data from an ultrasonic sensor with data from another sensor(e.g., a motion sensor), according to various embodiments. By combiningthe force determined by the ultrasonic sensor with a change inorientation or position, the determined force may be modified or theconfidence factor may be adjusted. Procedures of this method will bedescribed with reference to elements and/or components of variousfigures described herein. It is appreciated that in some embodiments,the procedures may be performed in a different order than described,that some of the described procedures may not be performed, and/or thatone or more additional procedures to those described may be performed.Flow diagram 1100 includes some procedures that, in various embodiments,are carried out by one or more processors (e.g., host processor 110 orsensor processor 172 of FIG. 1) under the control of computer-readableand computer-executable instructions that are stored on non-transitorycomputer-readable storage media. It is further appreciated that one ormore procedures described in flow diagram 1100 may be implemented inhardware, or a combination of hardware with firmware and/or software.

At procedure 1110 of flow diagram 1100, the force applied to theultrasonic sensor is received (e.g., the force determination of flowdiagram 900 of FIG. 9). At procedure 1120, a change in position and/ororientation is received from a motion sensor of the electronic device.The motion sensor may be an accelerometer or a gyroscope, and thedetected motion may be a linear acceleration or an angular acceleration,respectively. For example, with reference to FIG. 1, sensor 180-1 may bean accelerometer, and sensor 180-2 may be a gyroscope. The accelerometerdata and gyroscope data may also be combined in a fusion process.Operation of the motion sensors may be activated when the user startstouching the ultrasonic sensor, or when a first method determines aforce above a certain threshold, so that the motion sensor can be usedfor the remainder of the force process. In some embodiments, theaccelerometer may be active all the time, since it consumes less energy,and then the gyroscope, which consumes more energy, may be activatedselectively as just discussed based on detection of the finger or athreshold force.

At procedure 1130, the force determination is combined with theinformation received from the motion sensor. For example, if theelectronic device is equipped with a motion sensor (e.g., accelerometeror gyroscope of sensor 180 of FIG. 1), the pressing action of the usermay also lead to a change of position or orientation of the electronicdevice. The larger the pressing force of the user, the larger the changein position or orientation may be. The motion induced by the applicationof the force may depend on how the user is holding the device and wherethe sensor is positioned. The context of the information received fromthe motion sensor determines how the data is combined with the forcedetermination. The system may look for a predetermined type of motionpattern, rotation, or gesture. The system may be a learning system andlearn the correlation between the observed motion and the forcedetermined using the deformation. For example, if the electronic deviceis resting on a surface, information received from the motion sensor maybe disregarded. The context may be determined based on theaccelerometer, and the gyroscope may be selectively activated. Othersensors may also be used to determine the context, such as e.g.proximity sensors, light sensors, audio sensors, pressure sensors. Assuch, the motion data may have less weight in the initial learningstage, but as confidence is built, the weight of the motion data mayincrease in the final determination of the force.

At procedure 1140, the force determination may be modified and/or theconfidence of the force determination may be adjusted. It should beappreciated that the combination with motion sensor may not yieldadditional information is all situations. For example, if the user isholding the device in his or her hands while pressing the button achange in position/orientation may be detected, but if the device islying on a hard flat surface, no such change may be detected. Contextdetection may be used to determine if the additional (motion) sensordata may be combined with the ultrasonic data. The context detection maybe incorporated in the learning process described above.

In an embodiment of the invention, the effects of finger pressure on thesound propagation may be used to assess the pressure or forceindirectly. These indirect effects can be mechanical (e.g., soft layerscan be compressed at a point to which sound velocity, time-of-flight,and/ or interface impedance change in a measurable way), or thermal(e.g., in materials with high thermal expansion and low calorificcapacity and conductance, such as polymers, sound propagation willquickly change with finger pressure). Pressure and/or temperaturechanges in propagation layers may induce alteration to the ultrasonicsignals. These effects can be used as additional information forestimating thermal coupling with the finger, closely related to fingerpressure.

In embodiments of the invention where the ultrasonic sensor usespiezoelectric materials, the force applied by the user may cause adeformation of the piezoelectric material. If this deformation leads todetectable electric signals, these signals may also be used as a measurefor force. Again, a certain learning or calibration phase may beperformed to align these force measurements with other forcemeasurements discussed above if necessary.

In some embodiments, several of the force measurement methods above maybe combined based on the context and/or user. The contribution of thedifferent force measurements to the final determined force may depend onthe confidence of the determined force and the suitability of thedifferent methods to the context.

In addition to using the ultrasound fingerprint sensor to determine theidentity of the user and determine the force applied by the user, thesensor may also be used for navigation. In one embodiment, where theuser moves the finger across the surface of the sensor, the lateraldisplacement of the ridge/valley pattern across the field of view of thesensor may be used for navigation purposes. For example, to navigatebuttons, menus, moving a cursor on a screen, or controlling otherfunctions of the device. When the user applies little force, features ofthe finger (e.g., the ridge/valley pattern) are moved over the surfaceof the sensor without any distortion due to the lack of friction andadhesion of the finger to the surface. The threshold of force belowwhich the no deformation occurs depends on the surface materialcharacteristics, e.g., surface roughness.

In another embodiment, when the user applies a normal force into theultrasonic sensor and moves the finger, the surface of the finger maynot move with respect to the sensor due to static friction. In thissituation the user may still be able to move the finger, albeit withsmaller amplitude then when a small force is applied, but instead oftranslation of the finger, distortion or deformation of the finger isobserved. For example, the contact region of the finger changes withoutfeatures of the finger moving laterally relative to the surface of theultrasonic sensor. This lateral distortion may be used to determine howmuch the user moves the actual structure (e.g., bone) of the finger,while the skin remains immobile due to the static friction. Thedistortion of the finger may be determined using the techniquesdiscussed above to determine the compression and distortion due toforce, but in this case the lateral distortion or deformation ismeasured. In an intermediate force regime, the finger is moved in ajerking fashion due to the buildup and release of the static friction.In this case, the movement is a combination of deformation andtranslation, which may alternate in a temporal fashion.

FIGS. 12A and 12B illustrate cross section views of translation of afinger 1210 relative an example ultrasonic sensor 1230 at differentforces, according to some embodiments. FIG. 12A shows an example of thetranslation 1260 of finger 1210 of the user when little force is appliedand finger 1210 is moved laterally. The shape of the 1210 is notmodified (or slightly modified) during the lateral displacement 1250.FIG. 12B shows an example of the deformation 1262 of finger 1210 of theuser when a larger force is applied and finger 1210 is moved laterally.Due to the static friction, the surface of finger 1210 remains staticwith respect to the cover 1220 while structure 1240 of finger 1210(e.g., the bone) is moved. This causes the deformation or distortion ofthe shape of finger 1210 during the lateral displacement 1252.

During the low-force lateral displacement, the field of view of theultrasonic sensor may cover a continuously changing section of theridge/valley pattern of the finger. Therefore, the navigationfunctionality also is an opportunity to record a larger section of thefingerprint of the user than when the user just puts his or her fingeron the ultrasonic sensor in a static manner. Even when the user appliesa higher force and deforms the finger, the contact surface is increasedand may reveal a section of the fingerprint that is only available atlow force when the user rotates his or her finger on the ultrasonicsensor because these sections are normally on the side of the finger. Inshort, this means that both navigation modes may provide the opportunityto obtain a larger fingerprint than the surface of the ultrasonicsensor.

In accordance with various embodiments, the force determination and/orthe lateral movement of the finger, can be used to provide navigationfunctionality on a display of an electronic device communicativelycoupled to the ultrasonic sensor. In some embodiments, the sensor may beused in a dual navigation or displacement detection mode. This dual modemay comprise a first mode of finger translation when the user applieslittle force and the finger undergoes a translation, and a second modeof finger distortion when the user applies a large force and the fingeris distorted. The system may determine which mode the user is using, andmay react and/or process the displacement data differently. For example,in navigation applications the different modes may be coupled withdifferent navigation gains, where e.g. the low-force translation mode isused for large scale navigation and the high-force distortion mode isused for more precise small scale navigation (or vice-versa).Alternatively, one of the modes may be linked to cursor navigation andthe other mode may be linked to menu navigation. In another example, thelow-force mode may be used for navigation icons or menu items, and thelarge-force mode may be used for moving the items.

FIG. 13 illustrates a flow diagram 1300 of an example method for usingthe force to provide navigation functionality on a display of anelectronic device, according to various embodiments. By combining theforce determined by the ultrasonic sensor with a change in orientationor position, the determined force may be modified or the confidencefactor may be adjusted. Procedures of this method will be described withreference to elements and/or components of various figures describedherein. It is appreciated that in some embodiments, the procedures maybe performed in a different order than described, that some of thedescribed procedures may not be performed, and/or that one or moreadditional procedures to those described may be performed. Flow diagram1300 includes some procedures that, in various embodiments, are carriedout by one or more processors (e.g., host processor 110 or sensorprocessor 172 of FIG. 1) under the control of computer-readable andcomputer-executable instructions that are stored on non-transitorycomputer-readable storage media. It is further appreciated that one ormore procedures described in flow diagram 1300 may be implemented inhardware, or a combination of hardware with firmware and/or software.

At procedure 1310, the magnitude of the force is compared to athreshold. The purpose of this comparison is to determine if the fingeris static relative to the surface of the ultrasonic sensor or if thefinger is moving laterally relative to the surface of the ultrasonicsensor. At procedure 1320, it is determined whether the magnitude of theforce exceeds the threshold.

In one embodiment, if the magnitude of the force does not exceed thethreshold, as shown at procedure 1330 the ultrasonic sensor is used in afirst navigation mode, wherein the first navigation mode determinesmotion based at least in part on a translation of an outer surface ofthe finger relative to the ultrasonic sensor. The first navigation modeis also referred to herein as “motion mode.”

Motion mode is similar to the use of a touchpad. The motion of thefinger across the ultrasonic sensor is directly related to the motion ofthe cursor, and the position or change of position of the finger on theultrasonic sensor is linked to a position or change of position of acursor (or other virtual object) on a screen (as depicted in FIG. 12A).In other words, the motion vector of the cursor on the screen (MVC) isobtained by applying a gain k to the motion vector of the finger on thesensor (MVF): MVC=k*MVF. The gain may be constant, or may be dependenton the application. For example for applications that required largeamplitudes, the gain k may be set at a larger value, and forapplications that requires small precise movement, the gain k may be setat a smaller value. In addition, the gain may depend on the appliedforce, where a larger force may increase or decrease the gain asdesired. The force should not be so large as to prevent the moving ofthe finger across the surface. If the force is not used to determine thegain, and only the translation of the outer surface is relevant, thesensor may be adapted only the measure the outer surface. Because of thesmall size of the sensor, a desired cursor displacement with a largeamplitude may require multiple subsequent ‘swipes’ across the sensor bythe user. In between the multiple swipes of the finger, the cursormovement may be continued in the direction of the previous swipe toallow for a continuous cursor motion even though the multiple fingerswipes are of a discontinuous nature. This type of simulated movementmay not be constant but rather have a decaying velocity, which meansthat the cursor movement slow down and comes to a stop after the last ofthe series of swipes.

In one embodiment, if the magnitude of the force exceeds the threshold,as shown at procedure 1340 the ultrasonic sensor is used in a secondnavigation mode, wherein the second navigation mode determines motionbased at least in part on a direction and/or magnitude of an appliedforce. The second navigation mode may be selected from one of the twofollowing described modes, referred to herein as “position mode” and“arrow mode.” In one embodiment, it may be verified that an outersurface of the finger does not move substantially compared to thesensor, and a displacement threshold may be used. If the finger movesfurther than the displacement threshold, the system may exit the secondnavigation mode, and may enter the first navigation mode.

Position mode is similar to the use of a trackball or joystick. Toovercome the problems of larger amplitude movement as described in thefirst mode above, in this mode the motion of the cursor on the displayis based on the position of the finger, or position change of thefinger, on the ultrasonic sensor. In this mode, the initial positionwhere the finger touches the ultrasonic sensor is stored, and thedifference of position with this initial reference position is used tocontrol the cursor. The speed and direction of the motion of the cursoris deduced from the difference with the reference position. The largerthe difference with the initial position, the larger the speed of thecursor. A gain k may be applied between the vector PD expressing theposition difference and the motion vector of the cursor (MVC): MVC=k*PD.In addition, the gain may depend on the applied force, where a largerforce may increase or decrease the gain as desired. When the appliedforce is such that the surface of the finger does not move, and thefinger deforms laterally, as indicated in FIG. 12B, the lateral forcemay be used to determine the motion. The normal force may in addition beused to control the gain.

Arrow mode is similar to the position mode, but instead of applying again between the position difference and the motion vector of thecursor, a thresholding technique is applied. This means that if thedifference with the initial position exceeds a predefined threshold(which may depend on the direction), a dedicated function such asimitation of a keyboard arrow press is activated. The movement of thefinger with respect to the initial position can therefore mimic keyboardpresses, e.g., arrow presses. Alternatively, when a threshold positiondifference is surpassed, the cursor may be ‘launched’ in thecorresponding direction until a second event is detected that stops thestarted motion. This second event may be a motion in the oppositedirection or a change of the applied force. The speed of the launchedmotion may depend on the applied force.

The selection of a particular mode may be done based on the applied(initial) force. For example, a force below a predefined threshold mayoperate the fingerprint sensor in the first mode, while a force above acertain pressure may operate the fingerprint sensor in the second mode.Alternatively, the motion of the finger at the initial stages of thecontact with the sensor may determine the mode. A motion below a firstthreshold may select the second mode, and a motion above the firstthreshold may select the first mode. Additionally, a motion above asecond threshold, higher than the first threshold, may select the thirdmode where the detected motion is used to mimic a button press.Alternatively, the modes may be selected by e.g. tapping functions,certain finger gestures (e.g. swipe up), or may be defined depending onthe application or the user preferences. The selection of the modes inrelating to the force may also be determined by the context, theapplications, or the preferences of the user.

CONCLUSION

The examples set forth herein were presented in order to best explain,to describe particular applications, and to thereby enable those skilledin the art to make and use embodiments of the described examples.However, those skilled in the art will recognize that the foregoingdescription and examples have been presented for the purposes ofillustration and example only. Many aspects of the different exampleembodiments that are described above can be combined into newembodiments. The description as set forth is not intended to beexhaustive or to limit the embodiments to the precise form disclosed.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

Reference throughout this document to “one embodiment,” “certainembodiments,” “an embodiment,” “various embodiments,” “someembodiments,” or similar term means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment. Thus, the appearances of suchphrases in various places throughout this specification are notnecessarily all referring to the same embodiment. Furthermore, theparticular features, structures, or characteristics of any embodimentmay be combined in any suitable manner with one or more other features,structures, or characteristics of one or more other embodiments withoutlimitation.

What is claimed is:
 1. A method for determining force applied to asensor, the method comprising: receiving a plurality of sensor signalsfrom a sensor, the plurality of sensors signals from a fingerinteracting with the sensor; comparing a first data based at least inpart on a first sensor signal of the plurality of sensor signalscaptured at a first time with a second data based at least in part on asecond sensor signal of the plurality of sensor signals captured at asecond time; determining a deformation of the finger during interactionwith the sensor based on differences between the first data based atleast in part on the first sensor signal and the second data based atleast in part on the second sensor signal; and determining a forceapplied by the finger to the sensor based at least in part on thedeformation.